SCCunh 


AD- A 181  292 


T  SECURITY  CLASSIFICATION 


3a.  MCUAITV  classification  a ut nosit 


A  PERFORMING  ORGANIZATION  REPO 


•a  NAMt  OF  FSRFORMINO  ONOANIZATION 

'VERAC,  Inc. 


Sc.  AOORBSS  (013.  INN  «Ml  ZIP  Mil 

9605  Scranton  Road 
San  Diego,  CA  92121 


a*.  NAIM  OF  FUNDlNG/SPONSORING 
ONOANIZATION 

DARPA 


Sc.  AOON8SS  (CM?.  INN  Nil  ZIP  Co**) 

1400  Wilson  Boulevard 
Arlington,  VA  22209 


11.  TlTLt  Security  Clmmi  fiction) 

OCCAM  2nd  Qtr.  R&D  Status  Report 


ia'5r?*?ar1lAKos^o,#)  Prof.  Clark  Guest 


13a  TYPE  OF  NCFONT 

R&D  Status  Report 


is.  sufflsmsntanv  notation 


DOCUMENTATION  PAGE 

lift.  RESTRICTIVE  MARKINGS 

1  None 


3.  DlSTRISUTION/AVAILASILITV  OF  NSFONT 


S.  MONITONINO  ONOANIZATION  NSFONT  NUMSER(S) 

AFOSR-TR-  87-0  70  3 


7ft  NAIM  OF  MONITONINO  ORGANISATION* 

AFOSR 


7ft.  AODHSSS  (City,  INN  <M  ZIP  Cod e) 

AFOSR/Directorate  of  Elec  and  Mat  Sciences 
Building  410 

Boiling  AFB  DC  20332  -  6442 


S.  FNOCUNEMENT  INSTRUMENT  IDENTIFICATION  NUMSEN 

ARPA  order  5794/AFOSR  #F49620-S6-C-0070 


10.  SOUNCE  OF  FUNOINO  NOS. 


FNOONAM 
ELEMENT  NO. 


OFFICE  SVMSOL 
(If  omeliceblt) 


61/105P 


TASK 

WORK  UNIT 

NO. 

NO. 

"Si 

14.  DATE  OF  NEFpNT  (Yr.,  Mo..  Day )  I 


COSATI  COOES 


SUS.  ON. 


IS  SUBJECT  TERMS  (Con  turn*  on  revere*  If  ntetaaery  a nd  identify  by  block  number) 

bidirectional  associative  memory  (BAM),  LCLV,  fuzzy 
cognitive  map  (FCM),  neural  networks,  Lyapunov  functions 


IS-  ABSTRACT  (Comma,*  on  r*u*ra*  if  ntetaaary  ond  identify  by  block  number)  <y 

-’Found  appropriate  Lyapunov  or  ^energy"  function  for  continuous  BAMs.  Proved  that 
every  matrix  is  continuously  bidirectionally  stable,  generalizing  the  continuous  Hopfield 
model  and  extendigg  the  Cohen-Grossberg  Theorem  to  heteroassociativity.  Completed  paper 
^"Bidirectional  Associatie  Memories, ^submitted  to^IEEE  Trans  on  Sys,  Man,  Cyber.  Performec 
fuzzy  cognitive  map  (conceptual  computing)  knowledge  combination  experiments  on  UCSD 
neural  network  students.  Presented  BAM  and  FCM  theory,  as  invited  lectures,  to  UC  Berkeley 
Elec  Eng  Dept  and  USC's  Elec  Eng  Dept,  and  to  the  National  Science  Foundation. 

Performed  experiments  on  all-optical  Hughes  LCLV  (BAM)  associative  memory.  Fabric 
matrix  masks  to  represent  10  neurons.  Performed  computer  simulation  of  moment-term-expans' 
scale,  translation,  and  rotation  invariant  feature  extraction  algorithm  for  associative 
memory.  Began  optical  experiments  on ^counter  propagating*^  (competitive  learning)two-wave 
mixing  configuration  cast  in  barium  titinate.  Explorations  of  photorefractive  media  for 
associative  memory. _ 


30.  OlSTRISUTION/AVAl  LABILITY  OF  ABSTRACT 

UNCLASSl  PIS  D/UN  LIMITS  O  Q  SAME  AS  NFT.  □  OTIC  USERS  □ 


13ft  NAME  OF  NESFONSISLC  INDIVIDUAL 

Dr.  Lee  C.  Giles 


21.  ABSTRACT  SECURITY  CLASSIFICATION 

Unclassified 


22b  TELEPHONE  NUMBER 
(Include  Arra  Cod*) 

(202)  767  -  4984 


DO  FORM  1473. 83  APR 


EDITION  OF  1  JAN  73  IS  OBSOLETE. 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


I 


qiT» 


AFOSR-TH*  8  7-0703 


1.  Continuous  BAM.  The  bidirectional  associative  memory  (BAM) 
architecture,  developed  in  the  first  quarter  of  OCCAM,  was  extended  to 
arbitrary  continuous- valued  neurons.  We  proved  that  If  the  continuous  BAM 
system  is  given  by 
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where  M  Is  an  arbitrary  m-by-n  connection  matrix,  S  is  a  bounded  monotone 
Increasing  (e.g.,  sigmoid)  signal  function  ,  and  If  and  Jj  are  constant 
external  Inputs  to  a^  and  b j ,  then  the  appropriate  Lyapunov  or  "energy" 
function  E  for  the  network  is  given  by 
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This  proves  that  every  matrix  Is  continuously  bidirectionally  stable.  Among 
other  things,  this  result  trivially  generalizes  the  continuous  "Hopfield” 
model  when  M  -  M^.  More  fundamentally,  this  result  extends  the  Cohen- 
Grossberg  Theorem  for  unidirectional  autoassociators  with  fixed-weight 
symmetric  connection  matrices  C.  Now  (1)  Cohen-Grossberg  models  can  be  used 
for  heteroassociation,  and  (2)  when  they  are,  the  Cohen-Grossberg  Theorem  is 
valid  for  every  connection  matrix. 

2.  Fuzzv  Cognitive  Map  Experiment.  We  explored  conceptual  computing  by 
studing  the  fuzzy  cognitive  map  (FCM)  combination  technique  introduced  in 
the  first  quarter  of  OCCAM.  Bart  was  asked  to  guess  lecture  in  Robert 
Hecht-Ni el  sen's  neural  net  course  at  UCSD.  Students  received  a  lecture  on 
FCMs  and  related  technology.  As  a  mandatory  homework  assignment,  the 
students  were  given  a  sample  FCM  on  the  complex  subject  of  South  African 
politics.  The  students  were  asked  to  create  their  own  FCM  politico-causal 
interpretation  of  South  African  politics.  The  students  consisted  of  UCSD 
undergraduates  and  graduates,  UCSD  faculty,  and  San  Diego  industry 
participants. 

19  student  FCMs  were  combined  into  one  very  large  FCM  using  the 
simplext  combination  technique-equal  weighting  (all  wj  -  1)  for  each 
"expert.”  The  student  FCMs  had  on  average  8-12  nodes  and  15  -  25  causal 
Interconnections.  The  resulting  FCM  had  36  nodes  and  198  connections. 
These  causal  maps  were  manipulated  by  Bob  Sasseen  with  a  FCM  software 
package  he  developed  for  the  Texas  Instruments  Explorer  and  Symbolics  Lisp 
processors. 


Due  to  some  ambiguity  in  the  instructions  given,  some  students  wrote 
FCMs  with  fuzzy  weights  in  [-1,  1]  instead  of  the  requested  simple  bivalent 


weights  In  (-1,  1).  Of  course,  the  students  were  not  experts  on  South 
African  politics,  and  many  alleged  causal  connectlonsw  were  rather  dubious 
Indeed.  Yet,  as  expected  of  the  simple  additive  combination  technique,  most 
dubious  connections  had  little  or  not  weight  In  the  combined  map. 

Despite  these  problems,  the  combined  FCM  performed  very  well.  Strictly 
opposite  nodes-such  as  STRENGTH  OF  SOUTH  AFRICAN  ECONOMY  and  WEAKNESS  OF 
SOUTH  AFRICAN  ECONOMY,  or  VIOLENCE  and  SOCIAL  STABILITY- -almost  Invariably 
had  opposite  activation  values.  Running  the  FCM  showed  very  plausible 
development  of  the  situation  over  time. 

(Parenthetically,  the  consensus  FCM  was  rather  pessimistic  about  South 
Africa's  prospects.  Nearly  all  scenarios,  starting  with  various  Initial 
conditions,  showed  the  South  African  situation  steadily  deteriorating.) 

3.  BAM  Paper.  The  technical  paper  "Bidirectional  Associative  Memories"  was 
completed  and  submitted  for  publication  to  the  IEEE  Transactions  on  Systems. 
Man,  and  Cybernetics.  This  paper  Includes  the  OCCAM-developed  Temporal 
Associative  Memory  (TAM)  theory. 


4.  OCCAM  Lectures.  Bart  made  three  formal  (Invited)  presentations  during 
the  second  quarter  of  OCCAM:  (1)  "Fuzzy  Cognitive  Maps  and  Connectlonlst 
Architectures"  to  the  EECS  Department  at  UC  Berkeley  on  30  September,  (2) 
"Adaptive  Cogntlve  Processing"  to  the  National  Science  Foundation  at  Woburn, 
Massachusetts  on  8  October,  and  an  extended  version  of  "Adaptive  Cognitive 
Processing"  to  the  Electrical  Engineering  Department  at  USC  on  31  October. 
Judging  from  both  Immediate  audience  reaction  and  the  resultant  reprint 
requests  (over  120),  the  talks  were  well  received. 


During  the  second  quarter  of  OCCAM  at  UCSD,  concepts  developed  during 
the  first  quarter  moved  toward  Implementation.  Progress  was  made  In  the 
areas  of  an  all -optical  Liquid  Crystal  Light  Valve  (LCLV)  based  neural  net 
system.  Invariant  feature  extraction  for  neural  net  preprocessing,  and 
dynamic  photorefractlve  associative  memory  systems.  Details  of  these  three 
efforts  are  given  below. 

1.  LCLV  Neural  Net.  A  neural  net  system  based  on  the  Hughes  LCLV  was 
deslglned  during  the  first  quarter  of  OCCAM.  It  employs  fully  parallel 
optical  feedback,  uses  the  Intrinsic  sigmoid  response  of  the  light  valve, 
and  allows  dynamic  reprogramming  of  the  Interconnection  matrix  at  video 
frame  rates.  Ultimately,  such  a  system  could  Implement  a  network  of  500 
processing  elements  that  converge  to  steady  state  In  less  than  50  msec. 
Work  In  the  second  quarter  of  OCCAM  has  moved  toward  Implementation  of  a 
system  to  demonstrate  feasibility.  The  optical  system  was  designed  In 
detail,  parts  ordered,  received,  and  assembled.  Matrix  masks  were 
fabricated  to  Implement  a  10  neuron  system.  Final  alignment  of  the  system 
Is  underway  and  experiments  will  commence  early  In  the  third  quarter. 

In  the  process  of  designing  the  system,  a  novel  use  was  discovered  for 
an  existing  optical  element.  At  one  plane  of  the  system,  light  coming  from 
a  range  of  different  direction  must  be  Integrated  into  a  single  point  that 
Is  then  anamorphlcally  Imaged  through  the  rest  of  the  system.  A  diffusing 
screen  Is  used  for  this  purpose.  However,  the  broad  distribution  of  light 
that  occurs  on  the  output  side  of  the  diffuser  Is  difficult  to  capture,  even 


with  low  F-number  optics,  and  so  tho  usable  light  In  the  system  Is  greatly 
decreased.  It  was  found  that  a  fiberoptic  faceplate  served  to  combine  beams 
arriving  from  diverse  directions  and  produced  at  Its  output  face  an 
Intensity  sum  that  was  much  more  directed  along  the  normal  to  the  plate  than 
Is  typically  the  case  for  ground  glass.  While  there  Is  some  variation  light 
transmission  efficiency  with  Input  beam  angle,  wefound  It  to  be  quite 
acceptable  for  our  purposes.  This  linear  phase  cancellation  operation  Is 
new  among  existing  optical  devices  and  should  be  generally  applicable  In 
many  systems.  (A  fiber  optic  faceplate  Is  an  array  of  very  short  optical 
fibers  that  are  fused  together  with  their  axes  normal  to  the  plate  thus 
formed.  The  devices  were  available  to  us  as  cosmetic  rejects  from  the 
Hughes  LCLV  manufacturing  program.) 

L _ Invariant  Feature  Extraction  Algorithm.  A  scale,  rotation,  and 

translation  Invariant  feature  extract lonalgorlthm  was  developed  during  the 
first  quarter  of  OCCAM.  A  computer  simulation  produced  25  feature  moments 
of  chosen  sample  patterns,  In  this  case  letters  of  the  alphabet.  Continued 
simulation  during  the  second  quarter  has  allowed  a  subset  of  twelve  moments 
fromthe  Initial  25  to  be  chosen  that  provide  the  best  discrimination 
capability.  An  optical  correlation  system  using  computer-generated 
holographic  filters  has  been  designed  that  will  allow  simultaneous 
extraction  of  all  12  moments.  These  moments  will  then  serve  as  the  Inputs 
to  a  neural  net  that  will  be  trained  to  classify  scaled,  rotated,  and 
translated  Innputs.  Fabrication  of  the  required  holographic  filters  Is 
currently  underway.  Classification  networks  to  be  tried  Include  the 
contlnous  Cohen-Grossberg  (BAM),  back  propagation,  and  Kohonen's  "self- 
organizing"  system  (l.e.,  Grossberg's  1976  competitive  learning  model). 


3.  Photorefractlve  Associative  Memory.  Multiwave  mixing,  and  optical 
storage  In  photorefractlve  crystals,  have  been  Identified  by  a  number  of 
research  groups  as  potentially  useful  In  neural  net  applications. 
Fundamental  operations  that  trill  need  to  be  done  Include  Image 
amplification,  thresholding,  and  Incoherent-to-coherent  conversion.  At 
UCSD,  during  the  second  quarter  of  OCCAM,  we  began  experments  to 
characterize  these  effects  In  a  "counter  propagating"  two-wave  mixing 
configuration  done  In  barium  tltlnate.  ("Counter  propagation"  Is  the  name 
Hecht-Nlelsen  gave  his  Implementation  of  the  1976  Grossberg  competitive 
learning  model.  In  order,  apparently,  to  contrast  It  with  back  propagation.) 
The  "counter  propagating"  configuration  allows  beams  to  enter  the  crystal 
from  opposite  facesof  the  crystal,  thereby  recording  a  reflection  type 
hologram.  The  reflection  configuration  is  desirable  since  It  allows  greater 
use  of  the  volume  of  the  crystal,  and  It  removes  undlffractedllght  from  the 
system  during  hologram  readout.  Image  amplification  with  gains  of  up  to  30 
times  have  been  achieved,  and  much  higher  results  can  foe  expected. 

A  preliminary  system  design  using  photorefractlve  elements  has  been 
completed.  It  will  Implementf  an  attentional -orienting  system  of  the  type 
Introduced  by  Grossberg.  Such  systems  have  the  capability  to  adaptively 
code  arbitrary  Input  patterns,  l.e.,  no  pre-orthogonallzatlon  Is  necessary. 
Recall  can  use  time  sequential  pattern  matching  for  best-fit  classification. 


